Assisting People to Become Independent Learners in the Analysis of Intelligence

Section 1: What Makes Intelligence Analysis Difficult? A Cognitive Task Analysis of Intelligence Analysts by Susan G. Hutchins, Peter L. Pirolli, and Stuart K. Card; Section 2: Evaluation of a Computer Support Tool for Analysis of Competing Hypotheses by Peter Pirolli, Lance Good, Julie Heiser, Jeff Shrager, and Susan Huthins; Section 3: Collaborative Intelligence Analysis with CACHE and its Effects on Information Gathering and Cognitive Bias by Dorrit Billman, Gregorio Convertino, Jeff Shrager, J.P. Massar, Peter PirolliThe purpose of this project was to conduct applied research with exemplary technology to support post-graduate instruction in intelligence analysis. The first phase of research used Cognitive Task Analysis (CTA) to understand the nature of subject matter expertise for this domain, as well as leverage points for technology support. Results from the CTA and advice from intelligence analysis instructors at the Naval Postgraduate School lead us to focus on the development of a collaborative computer tool (CACHE) to support a method called the Analysis of Competing Hypotheses (ACH). We first evaluated a non-collaborative version of an ACH tool in an NPS intelligence classroom setting, followed by an evaluation of the collaborative tool, CACHE at NPS. These evaluations, along with similar studies conducted in coordination with NIST and MITRE, suggested that ACH and CACHE can support intelligence activities and mitigate confirmation bias. However, collaborative analysis has a number of trade-offs: it incurs overhead costs, and can mitigate or exacerbate confirmation bias, depending on the mixture of predisposing biases of collaborators.Office of Naval Researc

Effects of Sensemaking Translucence on Distributed Collaborative Analysis

Collaborative sensemaking requires that analysts share their information and insights with each other, but this process of sharing runs the risks of prematurely focusing the investigation on specific suspects. To address this tension, we propose and test an interface for collaborative crime analysis that aims to make analysts more aware of their sensemaking processes. We compare our sensemaking translucence interface to a standard interface without special sensemaking features in a controlled laboratory study. We found that the sensemaking translucence interface significantly improved clue finding and crime solving performance, but that analysts rated the interface lower on subjective measures than the standard interface. We conclude that designing for distributed sensemaking requires balancing task performance vs. user experience and real-time information sharing vs. data accuracy.Comment: ACM SIGCHI CSCW 201

Informavores: Active information foraging and human cognition

Just as the body survives by ingesting negative entropy, so the mind survives by ingesting information. In a very general sense, all higher organisms are informavores. The study of active information search is in the midst of a renaissance. Psychological research from diverse areas ranging from developmental psychology This symposium aims to bring together leading experts in this area to discuss how active information foraging can be understood from a diverse set of perspectives within cognitive science. Key themes include how prior knowledge influences search (Markant & Gureckis), how information and reward interact to determine choice (Meder & Nelson), developmental patterns in information seeking behavior (Nelson et al.), information foraging in complex sensemaking tasks (Pirolli), and the allocation of attention during statistical word learning (Yu). While each represents a distinct area of research, all discussants in the symposium share a core approach of applying computational models to understand information search in humans. The symposium should appeal to a broad set of attendees including educators, developmental psychologists, cognitive modelers, and computer scientists. The influence of priors on sequential search decisions - Doug Markant and Todd Gureckis Normative models of information acquisition predict that people's search decisions should be strongly influenced by their prior beliefs, which capture the set of alternative hypotheses they are considering. In the present experiments we tested whether people adjusted their information search behavior in response to sequential changes in the prior. Participants played a search game in which they had to identify the shape and location of multiple hidden targets in a display (similar to the board game Battleship). During the task they were told that the set of possible shapes had changed, and the key question was whether they would adjust their search decisions according to the predictions of a normative model. Manipulations of the prior included changes in the frequency of certain classes of targets as well as the introduction of higherorder constraints (e.g., that all targets would have the same shape). The results showed that an individual's prior could be recovered from their sequences of search decisions, but that there were notable differences in their ability to adjust to certain changes in the hypothesis space, an effect that is not predicted by the normative model. We discuss the implications of these findings for how people generate and represent hypotheses during the course of information foraging. Is people's information search behavior sensitive to different reward structures? -Björn Meder and Jonathan Nelson In situations where humans actively acquire information for classification, information search preferentially maximizes accurac

Making sense of actor behaviour: an algebraic filmstrip pattern and its implementation

Sense-making with respect to actor-based systems is challenging because of the non-determinism arising from concurrent behaviour. One strategy is to produce a trace of event histories that can be processed post-execution. Given a semantic domain, the histories can be translated into visual representations of the semantics in the form of filmstrips. This paper proposes a general pattern for the production of filmstrips from actor histories that can be implemented in a way that is independent of the particular data types used to represent the events, semantics and graphical displays. We demonstrate the pattern with respect to a simulation involving predators and prey which is a typical agent-based application

Sense-making strategies in explorative intelligence analysis of network evolutions

Visualising how social networks evolve is important in intelligence analysis in order to detect and monitor issues, such as emerging crime patterns or rapidly growing groups of offenders. It remains an open research question how this type of information should be presented for visual exploration. To get a sense of how users work with different types of visualisations, we evaluate a matrix and a node-link diagram in a controlled thinking aloud study. We describe the sense-making strategies that users adopted during explorative and realistic tasks. Thereby, we focus on the user behaviour in switching between the two visualisations and propose a set of nine strategies. Based on a qualitative and quantitative content analysis we show which visualisation supports which strategy better. We find that the two visualisations clearly support intelligence tasks and that for some tasks the combined use is more advantageous than the use of an individual visualisation

Managing Evidence in Food Safety and Nutrition

Evidence ('data') is at the heart of EFSA's 2020 Strategy and is addressed in three of its operational objectives: (1) adopt an open data approach, (2) improve data interoperability to facilitate data exchange, and (3) migrate towards structured scientific data. As the generation and availability of data have increased exponentially in the last decade, potentially providing a much larger evidence base for risk assessments, it is envisaged that the acquisition and management of evidence to support future food safety risk assessments will be a dominant feature of EFSA's future strategy. During the breakout session on 'Managing evidence' of EFSA's third Scientific Conference 'Science, Food, Society', current challenges and future developments were discussed in evidence management applied to food safety risk assessment, accounting for the increased volume of evidence available as well as the increased IT capabilities to access and analyse it. This paper reports on presentations given and discussions held during the session, which were centred around the following three main topics: (1) (big) data availability and (big) data connection, (2) problem formulation and (3) evidence integration. (C) 2019 European Food Safety Authority. EFSA Journal published by John Wiley and Sons Ltd on behalf of European Food Safety Authority

Foraging for foundations in decision neuroscience: insights from ethology

Modern decision neuroscience offers a powerful and broad account of human behaviour using computational techniques that link psychological and neuroscientific approaches to the ways that individuals can generate near-optimal choices in complex controlled environments. However, until recently, relatively little attention has been paid to the extent to which the structure of experimental environments relates to natural scenarios, and the survival problems that individuals have evolved to solve. This situation not only risks leaving decision-theoretic accounts ungrounded but also makes various aspects of the solutions, such as hard-wired or Pavlovian policies, difficult to interpret in the natural world. Here, we suggest importing concepts, paradigms and approaches from the fields of ethology and behavioural ecology, which concentrate on the contextual and functional correlates of decisions made about foraging and escape and address these lacunae